• Refine your search:
  • 
    Keyword
  • 
    Creator / Publisher:
    
    • Cotterell, Ryan (5)
    • Hall Maudslay, Rowan (5)
    • Pimentel, Tiago (3)
    • Valvoda, Josef (2)
    • Williams, Adina (2)
    • Beltagy, Iz (1)
    • Bethard, Steven (1)
    • Blasi, Damián (1)
    • Chai, Joyce (1)
    • Chakraborty, Tanmoy (1)
    •   more
  • 
    Year
  • 
    Medium
  • 
    Type
  • 
    BLLDB-Access

Search in the Catalogues and Directories

All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year Sort by creator [A → Z]' creator [Z → A]' publishing year ↑ (asc)' publishing year ↓ (desc)' title [A → Z]' title [Z → A]' Simple Search Hits 1 – 5 of 5 1 Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing Hall Maudslay, Rowan; Cotterell, Ryan In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2021) Abstract: Analysing whether neural language models encode linguistic information has become popular in NLP. One method of doing so, which is frequently cited to support the claim that models like BERT encode syntax, is called probing; probes are small supervised models trained to extract linguistic information from another model’s output. If a probe is able to predict a particular structure, it is argued that the model whose output it is trained on must have implicitly learnt to encode it. However, drawing a generalisation about a model’s linguistic knowledge about a specific phenomena based on what a probe is able to learn may be problematic: in this work, we show that semantic cues in training data means that syntactic probes do not properly isolate syntax. We generate a new corpus of semantically nonsensical but syntactically well-formed Jabberwocky sentences, which we use to evaluate two probes trained on normal data. We train the probes on several popular language models (BERT, GPT-2, and RoBERTa), and find that in all settings they perform worse when evaluated on these data, for one probe by an average of 15.4 UUAS points absolute. Although in most cases they still outperform the baselines, their lead is reduced substantially, e.g. by 53% in the case of BERT for one probe. This begs the question: what empirical scores constitute knowing syntax? URL: https://hdl.handle.net/20.500.11850/518986 https://doi.org/10.3929/ethz-b-000518986 BASE Hide details 2 Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing ... Hall Maudslay, Rowan; Cotterell, Ryan. - : ETH Zurich, 2021 BASE Show details 3 Speakers Fill Lexical Semantic Gaps with Context Blasi, Damián; Hall Maudslay, Rowan; Cotterell, Ryan... In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020) BASE Show details 4 A Tale of a Probe and a Parser Valvoda, Josef; Hall Maudslay, Rowan; Williams, Adina... In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020) BASE Show details 5 A Tale of a Probe and a Parser ... Hall Maudslay, Rowan; Valvoda, Josef; Pimentel, Tiago. - : ETH Zurich, 2020 BASE Show details

Mobile view All Catalogues UB Frankfurt Linguistik 0 IDS Mannheim 0 OLC Linguistik 0 UB Frankfurt Retrokatalog 0 DNB Subject Category Language 0 Institut für Empirische Sprachwissenschaft 0 Leibniz-Centre General Linguistics (ZAS) 0 Bibliographies BLLDB 0 BDSL 0 IDS Bibliografie zur deutschen Grammatik 0 IDS Bibliografie zur Gesprächsforschung 0 IDS Konnektoren im Deutschen 0 IDS Präpositionen im Deutschen 0 IDS OBELEX meta 0 MPI-SHH Linguistics Collection 0 MPI for Psycholinguistics 0 Linked Open Data catalogues Annohub 0 Online resources Link directory 0 Journal directory 0 Database directory 0 Dictionary directory 0 Open access documents BASE 5 Linguistik-Repository 0 IDS Publikationsserver 0 Online dissertations 0 Language Description Heritage 0